Exploiting Deep Semantics and Compositionality of Natural Language for Human-Robot-Interaction

We develop a natural language interface for human robot interaction that implements reasoning about deep semantics in natural language. To realize the required deep analysis, we employ methods from cognitive linguistics, namely the modular and compositional framework of Embodied Construction Grammar (ECG) [Feldman, 2009]. Using ECG, robots are able to solve fine-grained reference resolution problems and other issues related to deep semantics and compositionality of natural language. This also includes verbal interaction with humans to clarify commands and queries that are too ambiguous to be executed safely. We implement our NLU framework as a ROS package and present proof-of-concept scenarios with different robots, as well as a survey on the state of the art

Do Large Language Models know what humans know?

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a Large Language Model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans, nor does it explain the full extent of their behavior -- despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible

A pre-registered, multi-lab non-replication of the Action-sentence Compatibility Effect (ACE)

The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g., toward) matches the direction of the action in the to-be-judged sentence (e.g., Art gave you the pen describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.Fil: Morey, Richard. Cardiff University; Reino UnidoFil: Kaschak, Michael. Florida State University; Estados UnidosFil: Díez Álamo, Antonio. Universidad de Salamanca; España. Arizona State University; Estados UnidosFil: Glenberg, Arthur. Arizona State University; Estados Unidos. Universidad de Salamanca; EspañaFil: Zwaan, Rolf A.. Erasmus University Rotterdam; Países BajosFil: Lakens, Daniël. Eindhoven University of Technology; Países BajosFil: Ibáñez, Santiago Agustín. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Universidad de San Andrés; Argentina. University of San Francisco; Estados Unidos. Universidad Adolfo Ibañez; Chile. Trinity College Dublin; IrlandaFil: García, Adolfo Martín. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Universidad de San Andrés; Argentina. University of San Francisco; Estados Unidos. Universidad Nacional de Cuyo. Facultad de Educación Elemental y Especial; Argentina. Universidad de Santiago de Chile; ChileFil: Gianelli, Claudia. Universitat Potsdam; Alemania. Scuola Universitaria Superiore; ItaliaFil: Jones, John L.. Florida State University; Estados UnidosFil: Madden, Julie. University of Tennessee; Estados UnidosFil: Alifano Ferrero, Florencia. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Bergen, Benjamin. University of California at San Diego; Estados UnidosFil: Bloxsom, Nicholas G.. Ashland University; Estados UnidosFil: Bub, Daniel N.. University of Victoria; CanadáFil: Cai, Zhenguang G.. The Chinese University; Hong KongFil: Chartier, Christopher R.. Ashland University; Estados UnidosFil: Chatterjee, Anjan. University of Pennsylvania; Estados UnidosFil: Conwell, Erin. North Dakota State University; Estados UnidosFil: Wagner Cook, Susan. University of Iowa; Estados UnidosFil: Davis, Joshua D.. University of California at San Diego; Estados UnidosFil: Evers, Ellen R. K.. University of California at Berkeley; Estados UnidosFil: Girard, Sandrine. University of Carnegie Mellon; Estados UnidosFil: Harter, Derek. Texas A&m University Commerce; Estados UnidosFil: Hartung, Franziska. University of Pennsylvania; Estados UnidosFil: Herrera, Eduar. Universidad ICESI; ColombiaFil: Huettig, Falk. Max Planck Institute for Psycholinguistics; Países BajosFil: Humphries, Stacey. University of Pennsylvania; Estados UnidosFil: Juanchich, Marie. University of Essex; Reino UnidoFil: Kühne, Katharina. Universitat Potsdam; AlemaniaFil: Lu, Shulan. Texas A&m University Commerce; Estados UnidosFil: Lynes, Tom. University of East Anglia; Reino UnidoFil: Masson, Michael E. J.. University of Victoria; CanadáFil: Ostarek, Markus. Max Planck Institute for Psycholinguistics; Países BajosFil: Pessers, Sebastiaan. Katholikie Universiteit Leuven; BélgicaFil: Reglin, Rebecca. Universitat Potsdam; AlemaniaFil: Steegen, Sara. Katholikie Universiteit Leuven; BélgicaFil: Thiessen, Erik D.. University of Carnegie Mellon; Estados UnidosFil: Thomas, Laura E.. North Dakota State University; Estados UnidosFil: Trott, Sean. University of California at San Diego; Estados UnidosFil: Vandekerckhove, Joachim. University of California at Irvine; Estados UnidosFil: Vanpaeme, Wolf. Katholikie Universiteit Leuven; BélgicaFil: Vlachou, Maria. Katholikie Universiteit Leuven; BélgicaFil: Williams, Kristina. Texas A&m University Commerce; Estados UnidosFil: Ziv Crispel, Noam. BehavioralSight; Estados Unido

Evolution of the insecticide target Rdl in African Anopheles is driven by interspecific and interkaryotypic introgression.

The evolution of insecticide resistance mechanisms in natural populations of Anopheles malaria vectors is a major public health concern across Africa. Using genome sequence data, we study the evolution of resistance mutations in the resistance to dieldrin locus (Rdl), a GABA receptor targeted by several insecticides, but most notably by the long-discontinued cyclodiene, dieldrin. The two Rdl resistance mutations (296G and 296S) spread across West and Central African Anopheles via two independent hard selective sweeps that included likely compensatory nearby mutations, and were followed by a rare combination of introgression across species (from A. gambiae and A. arabiensis to A. coluzzii) and across non-concordant karyotypes of the 2La chromosomal inversion. Rdl resistance evolved in the 1950s as the first known adaptation to a large-scale insecticide-based intervention, but the evolutionary lessons from this system highlight contemporary and future dangers for management strategies designed to combat development of resistance in malaria vectors

Ambiguity in the Mind and the Lexicon

Words contain multitudes. This multiplicity of meanings raises two key questions, both of which this thesis attempts to address. First, are word meanings categorical or continuous? The results of Chapters 2-4 support a hybrid model, in which word meanings occupy a continuous state-space (Elman, 2009), which is further discretized along the boundaries of distinct senses. And second, does the amount and distribution of homophony in real lexica reflect a pressure to concentrate meanings in the most efficient, optimal wordforms? The results in Chapters 5-7 suggest that homophony can emerge without a direct pressure for efficiency––and further, that real lexica might select against homophones, particularly among the most frequent wordforms of a lexicon. This pressure could even explain other properties of human lexica, such as their large phonological neighborhoods

Ambiguity in the Mind and the Lexicon

Words contain multitudes. This multiplicity of meanings raises two key questions, both of which this thesis attempts to address. First, are word meanings categorical or continuous? The results of Chapters 2-4 support a hybrid model, in which word meanings occupy a continuous state-space (Elman, 2009), which is further discretized along the boundaries of distinct senses. And second, does the amount and distribution of homophony in real lexica reflect a pressure to concentrate meanings in the most efficient, optimal wordforms? The results in Chapters 5-7 suggest that homophony can emerge without a direct pressure for efficiency––and further, that real lexica might select against homophones, particularly among the most frequent wordforms of a lexicon. This pressure could even explain other properties of human lexica, such as their large phonological neighborhoods

Can a pressure against homophones explain phonological neighborhoods?

Words in human languages cluster together in phonological neighborhoods more closely than would be expected by chance. But why? One explanation is that large neighborhoods are directly selected for, possibly because they scaffold word learning and production. But it's also possible that they emerge as a byproduct of other constraints or selection pressures operating over real lexica. We advance one such selection pressure as a candidate explanation. A pressure to avoid overloading unique wordforms with homophones may lead to clusters of words that are not identical but similar. Using simulated baselines, we test the viability of this alternative account. We find that a pressure against loading too many meanings on unique wordforms––paired with the phonotactics of a target language––produces lexica with neighborhoods that are at least as large on average as those in real lexica. This does not rule out the possibility of a pro-neighborhood pressure, but it does demonstrate the viability of a parsimonious alternative account based on a pressure against homonymy for which there is independent evidence

Distrubutional Semantics Still Can't Account for Affordances

Can we know a word by the company it keeps? Aspects of meaning that concern physical interactions might be particularly difficult to learn from language alone. Glenberg & Robertson (2000) found that although human comprehenders were sensitive to the distinction between afforded and nonafforded actions, distributional semantic models were not. We tested whether technological advances have made distributional models more sensitive to affordances by replicating their experiment with modern Neural Language Models (NLMs). We found that only one NLM (GPT-3) was sensitive to the affordedness of actions. Moreover, GPT-3 accounted for only one third of the effect of affordedness on human sensibility judgments. These results imply that people use processes that go beyond distributional statistics to understand linguistic expressions, and that NLP systems may need to be augmented with such capabilities